Papers with backbone models

46 papers
Large Language Models are Good Annotators for Type-aware Data Augmentation in Grammatical Error Correction (2025.coling-main)

Copied to clipboard

Challenge: Large Language Models (LLMs) have demonstrated outstanding performance in many downstream tasks due to their emergent and in-context learning abilities.
Approach: They propose a method that considers LLMs as annotators for type-aware data augmentation in GEC tasks.
Outcome: The proposed method can generate consistent and typeaware data, which could improve the performance of large language models.
LACA: Improving Cross-lingual Aspect-Based Sentiment Analysis with LLM Data Augmentation (2025.acl-long)

Copied to clipboard

Challenge: Existing approaches to cross-lingual aspect-based sentiment analysis depend on translation tools.
Approach: They propose a cross-lingual aspect-based sentiment analysis framework that leverages a large language model to generate pseudo-labelled data in target language.
Outcome: The proposed approach outperforms translation-based approaches in six languages and five backbone models.
ReFSQL: A Retrieval-Augmentation Framework for Text-to-SQL Generation (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing methods that align natural language with SQL Language underestimate inherent structural characteristics of SQL and lead to structure errors.
Approach: They propose a retrieval-argument framework that aligns natural language with SQL Language and trains one encoder-decoder-based model to fit all questions.
Outcome: The proposed framework improves accuracy and robustness of text-to-SQL generation on five datasets.
Leveraging Multi-lingual Positive Instances in Contrastive Learning to Improve Sentence Embedding (2024.eacl-long)

Copied to clipboard

Challenge: Recent trends in learning monolingual and multilingual sentence embeddings are based on contrastive learning (CL) among an anchor, one positive and multiple negative instances.
Approach: They propose to leverage multiple positives to improve learning of multilingual sentence embeddings by using an anchor, one positive, and multiple negative instances.
Outcome: The proposed approach improves retrieval, semantic similarity, and classification performance on unseen languages.
Text-to-Code Generation with Modality-relative Pre-training (2024.eacl-long)

Copied to clipboard

Challenge: Large pre-trained language models have been applied to programming language tasks with great success, often through further pre-training of a strictly-natural language model.
Approach: They propose to map programming language modalities into the same embedding space by separating embeddable spaces between modality and modality-relative training objectives.
Outcome: The proposed model can be adapted and represented differently depending on which modality they belong to and to the ultimate benefit of the downstream task.
Beyond Coarse Labels: Fine-Grained Problem Augmentation and Multi-Dimensional Feedback for Emotional Support Conversation (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing ESC datasets often use coarse-grained problem categories, limiting models’ ability to address users’ complex, overlapping challenges.
Approach: They propose a generalizable fine-grained problem enhancement method that augments problem types, user scenarios, and profiles, enabling the construction of richer and more diverse ESC corpora.
Outcome: The proposed method improves both automatic and human evaluation metrics across different models.
Relation-Specific Attentions over Entity Mentions for Enhanced Document-Level Relation Extraction (2022.naacl-main)

Copied to clipboard

Challenge: Existing document-level relation extraction methods do not distinguish between mention-level features and entity-level feature . document-based methods are more challenging because of multiple mentions of entities.
Approach: They propose a method which selectively attentions different entity mentions with respect to candidate relations and performs relation-specific representations of entities.
Outcome: The proposed method improves relation-specific representations of entities on two benchmark datasets.
INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent (2025.acl-long)

Copied to clipboard

Challenge: Recent advances have underscored the potential of large language model (LLM)-based agents in financial decision-making.
Approach: They propose to evaluate LLM agents using 13 different LLMs as backbone models across various market environments and tasks.
Outcome: The proposed framework assesses the reasoning and decision-making capabilities of 13 different LLMs across various market environments and tasks.
Towards End-to-End Open Conversational Machine Reading (2023.findings-eacl)

Copied to clipboard

Challenge: Existing approaches to the problem of open-retrieval conversational machine reading (OR-CMR) use two separate modules to approach the problem's two successive sub-tasks.
Approach: They propose to model OR-CMR as a unified text-to-text task in a fully end-to end style and propose to use a text-based approach to solve the problem.
Outcome: Experiments on the ShARC and OR-ShARC dataset show that the proposed framework can generalize to different backbone models.
KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents (2025.findings-naacl)

Copied to clipboard

Challenge: Large Language Models (LLMs) fail to effectively guide the planning trajectories during task solving and result in planning hallucinations.
Approach: They propose a novel approach to enhance the planning capabilities of large language models by incorporating explicit action knowledge.
Outcome: The proposed approach can achieve comparable or superior performance to existing baselines on HotpotQA and ALFWorld.
Bridging the Code Gap: A Joint Learning Framework across Medical Coding Systems (2024.lrec-main)

Copied to clipboard

Challenge: Existing methods for automating medical coding focus on a single coding system . however, there are still challenges to overcome in coding.
Approach: They propose a joint learning framework for Across Medical coding systems which jointly learns different coding system through multi-task learning.
Outcome: The proposed framework improves the performance of the MIMIC-IV ICD-9 and MIMICIV I CD-10 datasets.
Adversarial Mixing Policy for Relaxing Locally Linear Constraints in Mixup (2021.emnlp-main)

Copied to clipboard

Challenge: Experiments on five text classification benchmarks and five backbone models have shown that our methods reduce the error rate over Mixup variants in a significant margin (up to 31.3%), especially in low-resource conditions (upto 17.5%).
Approach: They propose to add a small adversarial perturbation to the mixing coefficients rather than the examples to relax locally linear constraints.
Outcome: Experiments on five text classification benchmarks and five backbone models show that the proposed methods reduce the error rate over Mixup variants by 31.3%, especially in low-resource conditions.
Think Before You Speak: Cultivating Communication Skills of Large Language Models via Inner Monologue (2024.findings-naacl)

Copied to clipboard

Challenge: emergence of large language models (LLMs) improves capabilities of dialogue systems . but they lack communication skills, which make them more like information seeking tools .
Approach: They propose to empower LLMs with communication skills through inner monologues . they use a benchmark to evaluate the dialogue generation ability of the model .
Outcome: The proposed model outperforms the baselines in the evaluation of communication skills.
Transductive Learning for Textual Few-Shot Classification in API-based Embedding Models (2023.emnlp-main)

Copied to clipboard

Challenge: Proprietary and closed APIs are impacting the practical applications of natural language processing.
Approach: They propose a scenario where a pre-trained model is served through a gated API . they propose 'transductive inference' that leverages statistics of unlabelled data .
Outcome: The proposed model performs a few-shot classification task with unlabelled data using a gated API . the proposed model can be used to perform the task with a handful of classes .
FactCG: Enhancing Fact Checkers with Graph-Based Multi-Hop Data (2025.naacl-long)

Copied to clipboard

Challenge: Prior research on training grounded factuality classification models to detect hallucinations in large language models (LLMs) has relied on public natural language inference (NLI) data and synthetic data.
Approach: They propose a method that leverages multi-hop reasoning on context graphs extracted from documents to generate complex multi-level claims without relying on LLMs to decide data labels.
Outcome: The proposed model outperforms GPT-4-o on the LLM-Aggrefact benchmark with much smaller model size.
FS-Researcher: Test-Time Scaling for Long-Horizon Research Tasks with File-System-Based Agents (2026.acl-long)

Copied to clipboard

Challenge: Long trajectories in deep research often exceed model context limits, compressing token budgets for both evidence collection and report writing.
Approach: They propose a file-system-based framework that scales deep research beyond context window . a Context Builder agent acts as a librarian and a Report Writer agent composes the final report .
Outcome: Experiments on two open-ended benchmarks show that FS-Researcher achieves state-of-the-art report quality across different backbone models.
Self-Reasoning Language Models: Unfold Hidden Reasoning Chains with Few Reasoning Catalyst (2025.findings-acl)

Copied to clipboard

Challenge: Recent studies have demonstrated that inference-time scaling increases performance of Large Language Models (LLMs) in various reasoning tasks such as mathematics and complex question answering by increasing the length of Chain-of-Thought (CoT).
Approach: They propose a model which synthesizes longer CoT data and iteratively improves performance through self-training by incorporating a few demonstration examples.
Outcome: The proposed model achieves an average improvement of more than +2.5 points across five reasoning tasks: MMLU, GSM8K, ARC-C, HellaSwag, and BBH on two backbone models.
NG-Router: Graph-Supervised Multi-Agent Collaboration for Nutrition Question Answering (2026.eacl-long)

Copied to clipboard

Challenge: Existing methods for nutrition question answering face limited reasoning capacity and contextual overload . poor dietary patterns are associated with more than 11 million deaths in 2017 .
Approach: They propose a framework that enables supervised multi-agent collaboration for nutritional QA.
Outcome: The proposed framework outperforms single-agent and ensemble baselines in multi-agency reasoning tasks.
GuiLoMo: Allocating Experts and Ranks for LoRA-MoE via Bilevel Optimization with GuidedSelection Vectors (2025.findings-emnlp)

Copied to clipboard

Challenge: Low-Rank Adaptation (LoRA) methods are efficient for a large language model with reduced computational costs.
Approach: They propose a layer-wise expert numbers and ranks allocation strategy with GuidedSelection Vectors.
Outcome: The proposed method achieves superior or comparable performance to all baselines on three backbone models.
Key ingredients for effective zero-shot cross-lingual knowledge transfer in generative tasks (2024.naacl-long)

Copied to clipboard

Challenge: Existing studies have focused on zero-shot cross-lingual transfer . mBERT, mBART and mT5 provide high-quality representations for texts in various languages .
Approach: They propose to use mBART and NLLB-200 to finetune a multilingual pretrained language model on input-output pairs in one language and use it to make task predictions for inputs in other languages.
Outcome: The proposed approach significantly reduces generation in the wrong language with full finetuning and can be competitive in some cases.
Comments as Natural Logic Pivots: Improve Code Generation via Comment Perspective (2024.findings-acl)

Copied to clipboard

Challenge: Existing studies decompose complex tasks into intermediate steps by prompting strategies, such as Chain-of-Thought and its variants.
Approach: They propose to use code comments as natural logic pivot between natural language and code language to boost the code generation ability of code LLMs.
Outcome: The proposed method significantly improves the code pass rate on humanEval and MBPP, while the robustness of the logical comment decoding strategy is higher than the Chain-of-thoughts prompting.
Phrase-aware Unsupervised Constituency Parsing (2022.acl-long)

Copied to clipboard

Challenge: Recent studies have achieved inspiring success in unsupervised grammar induction using masked language modeling (MLM) as the proxy task.
Approach: They propose to regularize the parser with phrases extracted by an unsupervised phrase tagger to help the LM model quickly manage low-level structures.
Outcome: The proposed method improves the identification of high-level structures using phrase-guided masking.
Empowering Backbone Models for Visual Text Generation with Input Granularity Control and Glyph-Aware Training (2024.emnlp-main)

Copied to clipboard

Challenge: Existing text-to-image models struggle to generate images with legible visual texts . current models lack support for Chinese texts, misspelling, and lack of diversity .
Approach: They propose to empower backbone models to generate visual texts in Chinese and English . they propose to augment conventional training objective with glyph-aware training losses .
Outcome: The proposed methods can generate visual texts in English and Chinese while maintaining image generation quality.
Hallucination Diversity-Aware Active Learning for Text Summarization (2024.naacl-long)

Copied to clipboard

Challenge: Existing methods for alleviating hallucinations require costly human annotations . Existing approaches focus on a specific type of hallucinism, which limits their effectiveness .
Approach: They propose a method to detect hallucinations from errors in semantic frame, discourse and content verifiability in LLM summarization using HAllucination Diversity-Aware Sampling.
Outcome: The proposed framework reduces the need for costly human annotations to correct hallucinations in LLM outputs.
GhostBERT: Generate More Features with Cheap Operations for BERT (2021.acl-long)

Copied to clipboard

Challenge: Existing studies show that some parameters in pre-trained language models can be pruned away without severe accuracy degradation.
Approach: They propose a method which generates more features with very cheap operations from the remaining features and can be applied to unpruned BERT models to enhance their performance.
Outcome: Empirical results on the GLUE benchmark on three backbone models (i.e., BERT, RoBERTa and ELECTRA) verify the efficacy of the proposed method.
Retrieval-Generation Alignment for End-to-End Task-Oriented Dialogue System (2023.emnlp-main)

Copied to clipboard

Challenge: generative models struggle to distinguish subtle differences among retrieved knowledge records, resulting in suboptimal quality of generated responses.
Approach: They propose to use maximum marginal likelihood to train a perceptive retriever by utilizing signals from response generation for supervision.
Outcome: The proposed approach improves on three task-oriented dialogue datasets using T5 and ChatGPT as the backbone models.
SimOAP: Improve Coherence and Consistency in Persona-based Dialogue Generation via Over-sampling and Post-evaluation (2023.acl-long)

Copied to clipboard

Challenge: Existing work on large-scale corpora-based language models is limited and hard to generalize to all types of pre-trained language models.
Approach: They propose a two-stage SimOAP strategy that over-samples and post-evaluates large-scale responses from existing models and selects a good response based on multiple evaluation metrics.
Outcome: The proposed strategy outperforms baseline and automatic evaluation strategies in both automatic and human evaluations.
Adaptive Tool Use in Large Language Models with Meta-Cognition Trigger (2025.acl-long)

Copied to clipboard

Challenge: Existing research expands the tool arrays of large language models (LLMs), but the necessity of using these tools is often overlooked, leading to indiscriminate tool invocation.
Approach: They propose a meta-cognition proxy proxy for LLMs self-assessment of their capabilities, reflecting the model’s awareness of its own limitations.
Outcome: The proposed strategy is fine-tuned-free and costs minimal.
Attention-Enhancing Backdoor Attacks Against BERT-based Models (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing textual backdoor attacks focus on generating stealthy triggers or modifying model weights.
Approach: They propose a Trojan Attention Loss (TAL) which enhances the Trojan behavior by directly manipulating attention patterns.
Outcome: The proposed method improves the effectiveness of the backdoor attacks on different backbone models and tasks.
ReCUT: Balancing Reasoning Length and Accuracy in LLMs via Stepwise Trails and Preference Optimization (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing methods to train LLMs suffer from overthinking, leading to lengthy reasoning traces . Existing approaches to train large language models suffer from this problem .
Approach: They propose a method to combine multiple reasoning chains for training LLMs . they use stepwise exploration and long-short switched sampling to evaluate reasoning paths .
Outcome: The proposed method reduces reasoning lengths by approximately 30-50% . it also maintains or improves reasoning accuracy compared to baselines .
Skill-Based Few-Shot Selection for In-Context Learning (2023.emnlp-main)

Copied to clipboard

Challenge: Existing methods based on pre-trained embeddings can be easily biased by surface features that are not important for the target task.
Approach: They propose a skill-based few-shot selection method for in-context learning . it generates skill-specific descriptions for each test case and candidate example .
Outcome: The proposed method significantly outperforms existing methods in five cross-domain semantic parsing datasets and six backbone models.
Generative Table Pre-training Empowers Models for Tabular Prediction (2023.emnlp-main)

Copied to clipboard

Challenge: Existing methods to use table pre-training to boost tabular prediction performance remain open . a bachelor's degree earns less than 50K, and a generative LM can be used to unify tasks via one LM.
Approach: They propose a method that leverages table pre-training to empower tabular prediction models.
Outcome: The proposed method outperforms baseline models on 12 datasets and can be easily combined with various backbone models.
NeuReasoner: Towards Explainable, Controllable, and Unified Reasoning via Mixture-of-Neurons (2026.acl-long)

Copied to clipboard

Challenge: Existing Large Reasoning Models (LRMs) lack explainability and controllability . Existing models target isolated levels without unification, while relying on RL .
Approach: They propose an explainable, controllable, and unified reasoning framework driven by MoN.
Outcome: The proposed framework achieves performance gains of 27.0% while reducing token consumption by 19.6% 63.3%.
Beyond Meta-Reasoning: Metacognitive Consolidation for Self-Improving LLM Reasoning (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches for improving LLM reasoning remain episodic and lack reusable meta-reasoning skills.
Approach: They propose a framework that consolidates metacognitive experience from past reasoning episodes into reusable knowledge that improves future meta-reasoning.
Outcome: The proposed framework consolidates metacognitive experience from past reasoning episodes into reusable knowledge that improves future meta-reasoning.
Retrieval-Augmented Fine-Tuning With Preference Optimization For Visual Program Generation (2025.acl-long)

Copied to clipboard

Challenge: Visual programming languages (VPLs) allow users to create programs through graphical interfaces, which results in easier accessibility and widespread usage in various domains.
Approach: They propose to train VPLs from user instructions using large language models . they propose to use retrieval-augmented fine-tuning to leverage repetitive use of subroutines .
Outcome: The proposed method outperforms prompting-based methods for LD generation accuracy even with smaller backbone models.
Memory Matters More: Event-Centric Memory as a Logic Map for Agent Searching and Reasoning (2026.findings-acl)

Copied to clipboard

Challenge: Existing methods for storing and retrieving memory are limited by shallow semantic retrieval.
Approach: They propose a memory mechanism that organizes and retrieves past experiences to support decision-making.
Outcome: Experiments on LoCoMo and NarrativeQA show that CompassMem improves retrieval and reasoning performance across multiple backbone models.
FoE: Forest of Errors Makes the First Solution the Best in Large Reasoning Models (2026.acl-long)

Copied to clipboard

Challenge: Recent Large Reasoning Models (LRMs) have demonstrated remarkable success in complex reasoning tasks.
Approach: They propose a self-guided efficient reasoning framework that reduces FoE by pruning subs.
Outcome: The proposed model outperforms eight competitive baselines while reducing token consumption by 37.7% 70.4%.
Locomo-Plus: Beyond-Factual Cognitive Memory Evaluation Framework for LLM Agents (2026.acl-long)

Copied to clipboard

Challenge: Existing benchmarks and evaluation protocols focus on surface-level factual recall.
Approach: They propose a benchmark for assessing cognitive memory under cue–trigger semantic disconnect.
Outcome: The proposed framework reveals failures not captured by existing benchmarks.
SegDRE: A Salient Entity Guided Approach to Document-Level Relation Extraction (2026.findings-acl)

Copied to clipboard

Challenge: Existing models struggle to address two major bottlenecks in Document-level Relation Extraction: extreme class imbalance and complexity of multi-hop reasoning.
Approach: They propose a method that decouples the extraction space into dense and sparse scenarios.
Outcome: The proposed approach yields consistent improvements over various backbone models and achieves advanced performance compared to existing enhancement methods.
Enhance Multimodal Consistency and Coherence for Text-Image Plan Generation (2025.findings-acl)

Copied to clipboard

Challenge: Existing studies on textual plan generation only focus on LLMs, enabling applications in robotics, virtual assistants, and instruc.
Approach: They propose a framework that generates and refines text-image plans step-by-step . they collect a new benchmark consisting of 1,100 tasks and their text- image pair solutions covering 11 daily topics.
Outcome: The proposed framework generates and refines text-image plans step-by-step and improves on existing models.
Empowering GraphRAG with Knowledge Filtering and Integration (2025.emnlp-main)

Copied to clipboard

Challenge: Large language models suffer from knowledge gaps and hallucinations, resulting in incorrect or poor reasoning.
Approach: They propose Graph retrieval-augmented generation (GraphRAG) which integrates structured knowledge from external graphs to enhance model's reasoning.
Outcome: Experiments on knowledge graph QA tasks show that GraphRAG significantly improves reasoning performance across multiple backbone models.
Bone Soups: A Seek-and-Soup Model Merging Approach for Controllable Multi-Objective Generation (2025.acl-long)

Copied to clipboard

Challenge: Existing approaches focus on merging language models tuned on single objectives . existing approaches ignore the impacts of competing objectives on model tuning .
Approach: They propose a model merging approach that seeks a series of backbone models and merges them according to user preferences.
Outcome: The proposed approach exhibits strong controllability and Pareto optimality in controllable multi-objective generation.
GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment (2026.acl-long)

Copied to clipboard

Challenge: Large Language Models (LLMs) have achieved remarkable performance across NLP tasks . however, in long-context scenarios, they face high computational cost and information redundancy.
Approach: They propose an encoder-decoder context compression framework that generates a compact sequence of soft tokens for downstream tasks.
Outcome: Experiments show that GMSA outperforms baselines on multiple long-context question answering and summarization benchmarks while maintaining low end-to-end latency.
Translation with Thought: Difficulty-Adaptive Reasoning via Reinforcement Learning for Multi-Domain Machine Translation (2026.acl-long)

Copied to clipboard

Challenge: Multi-domain machine translation (MDMT) is a unique challenge due to varying levels of linguistic complexity across domains.
Approach: They propose a resource-rational framework that learns to modulate inference between intuitive and deliberate reasoning.
Outcome: Evaluated on 15 benchmarks spanning in-domain and out-of-domain settings, as well as 3 seen and 59 unseen languages, with ablations across three backbone models, TwT-7B and Twt-14B outperform much larger SOTA reasoning models in translation quality, while reducing token usage by 32–60%.
LearnAct: Few-Shot Mobile GUI Agent with a Unified Demonstration Benchmark (2026.findings-acl)

Copied to clipboard

Challenge: Mobile GUI agents show promise in automating tasks but face significant generalization challenges in long-tail scenarios.
Approach: They propose a benchmark framework for mobile GUI agents that measures the performance of GUI agents by analyzing their performance.
Outcome: The LearnGUI benchmark outperforms existing methods in offline and online evaluations and demonstrates consistent gains across model architectures.
CRISP: Compressing Redundancy in Chain-of-Thought via Intrinsic Saliency Pruning (2026.findings-acl)

Copied to clipboard

Challenge: Existing approaches to compress CoT with external compressors fail to align with the model’s internal reasoning dynamics, resulting in the loss of critical logical steps.
Approach: They propose a framework that exploits the model’s intrinsic saliency to compress CoT by exploiting its reasoning termination token .
Outcome: The proposed framework reduces redundancy in reasoning chain by exploiting the model’s intrinsic saliency.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations